Cloud computing has introduced the concept of service orchestration. Service orchestration involves the following functions:    1) Placing and moving virtual machines (VMs) in particular locations that optimize the use of resources, such as server power consumption, bandwidth to the Internet, etc.    2) Starting, stopping, and pausing VMs that implement the service.    3) Arranging for network connectivity to the Internet and between the VMs implementing the service.    4) Arranging for load balancing by scheduling a load balancer to distribute the load and starting up additional service VMs if the load requires it.    5) Performing other functions involved in managing service VMs, their access to storage, and access networking.
The services “orchestrated” by cloud orchestration tools are typically end user services, like Web applications, ecommerce applications, etc.
Another class of services is network services. Network services perform operations on flows of packets in real time prior to the delivery of these packets to the end user or to end user services. Examples of such applications include load balancers for ensuring traffic loads are spread among service VMs, deep packet inspection (DPI) to check for flows that have security problems, firewalls to exclude particular kinds of traffic, media gateways for audio/video traffic, and Web proxies of various sorts, etc. These services often have real time performance requirements since they interpose on the delivery of packets between the source and destination, so a person is typically waiting for a response on the destination side. Often a chain of such services must be run on a packet flow, and different services must be run on different packet flows. For example, if a packet flow is identified as a YouTube video, DPI need not be run on it since the content is known and does not pose a security risk, but perhaps an ad insertion service might.
Present solutions to the network services orchestration problem involve one of three approaches:    1) If the network service is a hardware appliance, like a hardware load balancer, the software implementing management of the service may be delivered together with the hardware.    2) Managing network services as part of end user services orchestration. An example is including a load balancer VM as part of an end user service orchestration.    3) Requiring the network services to be virtual appliances and routing packets between them using encapsulation. Sometimes the service VMs must be wrapped in specialized wrappers. Orchestration is then possible for the virtualized appliances.
The problem with the first solution is that it does not allow for multiple services in the service chain. If the orchestration is tied to a single service, like load balancing, it becomes very difficult to include other services in the service chain. Complex routing solutions using access control lists (ACLS) must be programmed into the switches and routers, and if another service is a hardware service, rearranging the routing to reach the location of the hardware service may require manual intervention.
The problem with the second solution is that some network services are either not of interest to the designer of an end user service or need to be inserted by default. For example, the designer of an end user service should not need to be aware that DPI is running on their traffic, and therefore should not need to be responsible for including DPI in their service orchestration. The DPI service is both to protect the end user service and to protect the network operator.
The problem with the third solution is that it does not allow for hardware implementations for network services, nor for implementations that run on the bare metal with system software optimized for high performance packet forwarding because the services must run in VMs, and in some instances VMs packaged with specific agents. Because the services must be packaged as VMs, hardware-based services or services running on bare metal with optimized system software for packet processing are not usable.
What is needed, therefore, is a clean network services orchestration solution that can optimally place and manage a chain of network services, the ordering of which can differ depending on the particular flows that need processing, and can arrange for flows to be routed through the services in the proper order.